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" The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). ^ 

Status 

1 )^ Responsive to comnnunication(s) filed on 25 January 2002 and 05 October 1999 . 
2a)n This action is FINAL. 2b)K This action is non-final. 

3) 0 Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11 , 453 O.G. 213. 
Disposition of Claims 

4) K Claim(s) 1-48 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) S Claim(s) 1-48 is/are rejected. 

Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)n accepted or bjD objected to by the Examiner. 

Applicant may not request that any objection to the drawlng(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) 0 The proposed drawing correction filed on is: a)^ approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAII b)D Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
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DETAILED ACTION 



Claim Rejections - 35 USC § 112 



1 . The following is a quotation of the second paragraph of 35 U.S.C, 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

2. Claims 19-20 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. Claim 19 recites "the current presentation time of the media content" on line 4. 
There is insufficient antecedent basis for this limitation in the claims. 

Claim Rejections - 35 USC §102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 



4. Claims 1, 3, 10-12, 14, 23, 27-33, 36, 39-42, and 44-46 are rejected under 35 
U.S.C. 102(e) as being anticipated by Eberman et al ("Eberman", US 6,173,287 Bl). 

As per independent claim 1, Eberman teaches a networked client/server system, 
comprising: 

a network annotation server (fig. 1 A; col. 2, Unes 35-36; indexed database server 32); 
a network media server (fig. 1 A; media database server / library 28); 
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a client that communicates with both the annotation server and the media server over a 
data communications network (fig. 1 A; browser client 20); 

muhimedia content available from the media server (col. 2, Unes 36-37); 

a plurality of annotations, corresponding to the multimedia content, available from the 
annotation server, each of the plurality of annotations including annotation content and a 
temporal range identifier that identifies a segment of the multimedia content to which the 
annotation corresponds (fig. 10; col. 17, lines 1-48); and 

wherein the client supports a graphical user interface that presents a plurality of 
annotation identifiers corresponding to the multimedia content and that enables a user to request 
selected ones of the plurality of annotations, based on the plurality of annotation identifiers, and 
to render the requested annotations (col. 8, lines 18-32). 

As per independent claims 3 and 10-11, Eberman teaches a graphical user interface for 
adding annotations to an aimotation database from a network client, the graphical user interface 
comprising: 

an annotation content field via which a user can enter content for a new annotation to the 
network client; and an annotation type selector presenting a plurality of annotation media types, 
whereupon selection of one of the plurality of annotation types causes the network client to 
change presentation of the annotation content field based on the selected annotation type; an add 
button, whereupon selection of the add button causes the network client to forward the content 
from the annotation content field to an annotation server to be added to an annotation database; 
and temporal range information identifying a segment of media content to which the annotation 
corresponds (figs. 8-10; col. 16, lines 1 1 - 67). 
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Eberman does not expressly disclose an annotation content field, an annotation type 
selector, an add button and temporal range information in a GUI as claimed. However, these 
GUI elements would have been inherently part of Eberman' s GUI so as to provide users with a 
means for entering the annotation information as depicted in the database tables as shown in 
figures 8-10. 

As per claims 12 and 14, Eberman teaches a graphical user interface at a network client 
to search for annotations corresponding to media content in an annotation database, the graphical 
user interface comprising: 

an annotation set selector via which a user can identify one or more of a plurality of 
aimotation sets to be searched; a search criteria portion via which a user can identify search 
criteria; and a query button, wherein actuation of the query button causes the network client to 
forward the identified one or more annotation sets and the identified search criteria to an 
annotation server to search the annotation database; and wherein the search criteria portion 
includes one or more of: an annotation creation date entry field, a keyword entry field, and a 
temporal range entry field (col. 8, lines 18-32). 

Claims 23 and 27 are similar in scope to claim 10, and are therefore rejected under 
similar rationale. 

Claim 28 is similar in scope to claim 27, and is therefore rejected under similar rationale. 
As per claim 29, Eberman teaches the data associated the user interface comprises an 
annotation set identifier (col. 7, lines 46-51; object identification number). 

Claim 30 is similar in scope to claim 1 1, and is therefore rejected under similar rationale. 
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As per claim 31, Eberman teaches one or more computer-readable memories containing a 
computer program that is executable by a processor to perform the method recited in claim 28 
(abstract). 

Claims 32, 36 and 39-40 are similar in scope to claim 30, and is therefore rejected under 
similar rationale. 

Claim 33 is similar in scope to claim 30, and is therefore rejected under similar rationale. 

Claim 41 is similar in scope to claim 31, and is therefore rejected under similar rationale. 

Claim 42 is similar in scope to claim 32, and would have been therefore rejected under 
similar rationale. Eberman further teaches the presenting of a plurality of likely temporal 
locations of the media content to associate the new annotation with (figs. 12-13; col. 15, lines 4- 
11). 

As per claims 44-45, Eberman teaches for each of the plurality of likely temporal 
locations, displaying a visual indication of the likely temporal location, wherein the visual 
indication includes one or more of: a video frame of the media content, a numerical presentation 
time of the media content, and an indicator on a graphical time bar (figs 12-13). 

Claim 46 is similar in scope to claim 31, and is therefore rejected under similar rationale. 

Claim Rejections - 35 USC §103 
5. The. following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill 
in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 
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6. Claims 4-6, 8, 13, 15-22, 24, 26, 34-35, 37-38, and 47-48 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Eberman et al. ("Eberman", US 6,173,287 Bl). 

As per claims 4-6, by disclosing that "annotations could be, for example, a piece of 
information relating to data in a particular form such as, for example audio or video data" (col. 2, 
lines 29-31), it would have been obvious to an artisan that Eberman teaches the plurality of 
annotation types to include audio, video, and URL, as well as suggests text to be an option, 
wherein the annotation content field comprises a uniform resource locator (URL)/text field when 
an URL/text type is selected respectively. Text as an option would provide an important form 
for all annotations of multimedia contents. 

As per claim 8, although Eberman teaches a plurality of annotation type selector, 
Eberman does not explicitly disclose the type selector to comprise a radio button for each of the 
plurality of annotation types. Official Notice is given that the use of radio button as a means for 
selecting an item from a list of items in a menu is well known in the art. It would have been 
obvious to an artisan at the time of the invention to combine the use of radio buttons with 
Eberman GUI in order to provide users with an effective means for selecting an item that is 
mutually exclusive from a list of items in a menu. 

As per claim 13, Eberman teaches the GUI to comprise of a control, whereupon selection 
of the control causes the network client to add, as an additional search criteria, a media content 
identifier (fig. 11; col. 9, lines 12-31; and col. 21, lines 8-9). Although Eberman uses a button 
instead of a check box, but since they are equivalent in fimction, it would have been obvious to 
an artisan at the time of the invention that the implementation of either one of these features 
would have been an implementation choice of preference. 
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As per claim 15, Eberman teaches the GUI to comprise a detail level selector via which a 
user can indicate an amount of data to be displayed for each annotation that matches the search 
criteria (col. 22, lines 63-67). 

As per claims 16-18, Eberman teaches a graphical user interface for viewing, at a 
network client, annotations corresponding to media content, the graphical user interface 
comprising: an annotation identifier list via which an identifier for each of a plurality of 
aimotations corresponding to the media content is displayed, the identifier including an 
indication of a type of content included in the annotation; an actuation mechanism to enable a 
user to select one of the annotation identifiers; wherein the type of content includes one or more 
of: audio content, text content, video content, and uniform resource locator (URL) content; and 
wherein the identifier for an annotation includes one or more of: an indication of an author of the 
annotation, an indication of an annotation set that the annotation belongs to, an indication of a 
date the annotation was created, and a summary of the annotation (fig. 12; col. 22, lines 23-46). 

Eberman does not explicitly disclose the selection of one of the annotation identifiers to 
cause the network client to highlight the annotation identifier. Official Notice is given that 
highlighting an item as a result of the item being selected is well known in the art. It would have 
been obvious to an artisan to include this feature with Eberman' s GUI in order to provide users 
with a means for distinguishing a selected identifier from the unselected identifiers. 

As per claim 19, Eberman teaches a mechanism to identify a particular identifier that 
corresponds to an annotation of the plurality of annotations with a temporal range having a 
beginning time closest to the current presentation time of the media content (col. 8, line 33 - col. 
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9, line 31; and col. 17, line 49 - col. 18, line 16; the subjected identifier would be the very first 
identifier shown on the results query page). 

As per claim 20, Eberman does not disclose an arrow being used to indicate the particular 
identifier. Official Notice is given that the use of a visual icon associated with an item in a GUI 
is well known in the art. It would have been obvious to an artisan at the tune of the invention to 
combine the use an icon such as that of an arrow with Eberman' s GUI to further provide users to 
quickly identify a particular identifier through such a visually means. 

As per claim 21, Eberman teaches the GUI to further comprise a preview portion via 
which annotation content for a selected one of the plurality of annotations is displayed (col. 22, 
lines 47-50). 

As per claim 22, Eberman does not disclose a menu including a plurality of options 
identifying criteria to be used to order the annotation identifiers in the annotation identifier 
portion, whereupon selection of one of the plurality of options by a user causes the network 
client to arrange the annotation identifiers in the annotation identifier list in accordance with the 
criteria of the selected option. Official Notice is given that sorting annotation identifiers 
according to a preference options is well known in the art. It would have been obvious to an 
artisan at the time of the invention to include this feature with Eberman' s GUI in order to 
provide users with different sorting choices. 

Claims 24 and 26 are similar in scope to claims 19 and 21 respectively, and are therefore 
rejected under similar rationale. 

As per claims 34-35, Eberman does not specifically disclose the use of a means to 
indicate acceptance or rejection while in the process of associating the new annotation with a 
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temporal location of the media content. Official Notice is given that the use of a confirmation 
means, such as <0K> and <CANCEL>, at the end of a process is well known in the art. It 
would have been obvious to an artisan at the time of the invention to include such a means with 
Eberman's method in order to allow users with a chance to reconsider before committing the 
new annotation with a temporal location of a portion of the media content, or to associate the 
new annotation with another different portion of the media content. 

As per claims 37-38, Eberman does not explicitly disclose the presenting to comprise 
displaying a video frame or a presentation time of the media content corresponding to the likely 
temporal location. Official Notice is given that the displaying of video frames and presentation 
time corresponding to a media location being reviewed for annotation is well known in the art. It 
would have been obvious to an artisan at the time of the invention to include such displays with 
Eberman' s method in order to provide users with a visual means for verifying pertinent 
information about the media content being considered for the new annotations. 

As per claims 47-48, Eberman teaches one or more computer-readable media having 
stored thereon a computer program that, when executed by one or more processors, causes the 
one or more processors to perform fimctions including: 

presenting a graphical user interface at a client computer, the graphical user interface 
enabling a user to add a new annotation corresponding to media content; receiving a user request 
to add the new annotation; identifying a user request time that is a time, with reference to the 
presentation time of the media content, that the user request is received; and selecting, based on 
the user request time, a presentation time of the media content to associate the new annotation 
with, wherein the presentation time is a different time than the user request time, wherein the 
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selecting comprises selecting a begin time and an end time of the media content to define a 
segment of the media content to associate the new annotation with, and wherein the begin time is 
prior to the user request time and the end time is subsequent to the user request time (col. 8, line 
18 - col. 9, line 31). While Eberman does not explicitly disclose the user request time with a 
starting and ending time codes to be a different time than the presentation time which consists 
also of a beginning and ending time, it would have been obvious to an artisan at the time of the 
invention that there would have been an instance where the query returns a media content with a 
presentation a time frame closest to the user requested time relative to both the starting and 
ending time. In such an instance, the user request time may be an inclusive portion (slice) of the 
media content, which would make the presentation begin time is prior to the user request time 
and the presentation end time is subsequent to the user request time, 

7. Claims 2 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Eberman 
et al. ("Eberman", US 6,173,287 Bl) in view of Sutton et al. ("Sutton", US 4,649,499). 

As per claim 2, although Eberman teaches the graphical user interface to include a 
plurality of option buttons, whereupon selection of one of the plurality of the option buttons, an 
associated action corresponding to the selected button is taken (fig. 11; buttons 174-178), 
Eberman does not teach the buttons to be user-configurable. Sutton teaches a GUI, wherein a 
series of user-configurable buttons associated with respective user-defined actions are provided 
(col. 3, lines 44-46). It would have been obvious to an artisan at the time of the invention to 
combine Sutton's teaching with Eberman's system in order to further provide more flexibility to 
user's capability in defining selectable options according to preference. 

Claim 25 is similar in scope to claim 2, and is therefore rejected under similar rationale. 
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8. Claims 7 and 43 are rejected under 35 U.S.C. 103(a) as being unpatentable over Eberman 
et al. ("Eberman", US 6,173,287 Bl) in view of King et al. ("King", US 5,600,775). 

As per claim 7, Eberman does not disclose the aimotation content field to comprise, when 
an audio type is selected, a plurality of audio controls including one or more of: record, stop, 
pause, play, fast forward, and rewind. King teaches a method for annotating multimedia 
comprising a GUI comprising graphical elements such as buttons which provide control 
functions for media annotation such as record, stop, pause, play, fast forward, and rewind (fig. 
2). It would have been obvious to an artisan at the time to combine King's control functions 
with Eberman's GUI in order to provide a means for controlling annotating functions. 

As per claim 43, although Eberman teaches the presenting the plurality of likely temporal 
locations to comprise identifying a different one of the plurality of likely temporal locations to 
the user, Eberman does not teach the use of a rewind button to be actuated each time a different 
temporal location is to be identified. King teaches a rewind button for media annotation (fig. 2; 
button 53). It would have been obvious to an artisan at the time to combine King's rewind button 
with Eberman' s method in order to provide a means for changing to a different temporal location 
as a likely candidate to be associated with the new annotation. 

9. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Eberman et al. 
("Eberman", US 6,173,287 Bl) in view of Hou et al. ("Hou", US 5,838,313). 

As per claim 9, although Eberman teaches the use of an email feature (col. 9, lines 50- 
61), Eberman does not explicitly disclose an email field to identify a recipient to receive an email 
notification of the new annotation. Hou teaches a multimedia system wherein a dynamic 
annotation handler allows a user to record/playback annotations from the user input as well as 
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forwarding the annotations to other individuals via emails (col. 2, line 56 - col. 3, line 13). It 
would have been obvious to an artisan at the time to combine Hou's emailing feature with 
Eberman's GUI in order to provide a means for forwarding/informing pertinent recipients of the 
new annotations. 

Information Disclosure Statement 

10. The information disclosure statement filed 11/21/2000 fails to comply with 37 CFR 
1.98(a)(2), which requires a legible copy of each U.S. and foreign patent; each publication or that 
portion which caused it to be listed; and all other information or that portion which caused it to 
be listed. It has been placed in the application file, but the information referred to therein has not 
been considered. 

Conclusion 

11. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

deVries et al. (US 6,332,144 Bl) teaches a technique for annotating media. 

deVries et al. (US 6,311,189 Bl) teaches a method for matching a query to a portion of 

media. 

Matsuzawa et al. (US 6,085,185) teaches a retrieval method for contents from multimedia 
database. 

Davis et al. (US 5,969,716) teaches a time-based media processing system. 
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Weber et al. (US 5,893,110) teaches a method for automatically constructing a database 
query for data assets associated with muhimedia works. 

Jain et al. (US 6,360,234 B2) teaches a video cataloger system with synchronized 



Boreczky et al. (US 6,366,296 Bl) teaches a media browser using multimodal analysis. 

Kisor (US 5,809,250) teaches methods for creating and sharing replayable modules 
representative of web browsing session. 

Anderson (US 5,583,980) teaches a time-synchronized annotation method. 

Pavley et al. (US 6,317,141 Bl) teaches a method for editing heterogeneous media 
objects in a digital imaging device. 



12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sy Luu whose telephone number is (703) 305-0409. The 
examiner can normally be reached on Monday - Thursday from 6:30 am to 4:00 pm (EST). The 
examiner can also be reached on alternate Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 

The fax number for the organization where this application or proceeding is assigned are 
as follows: 

(703) 746-7238 [After Final Communication] 

(703) 746-7239 [Official Communication] 

(703) 746-7240 [For status inquiries, Draft Communication] 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptipnist whose telephone number is (703) 305-3900. 



encoders. 



Inquires 




SyD.M 
Patent Examiner 
April 17, 2002 



